The present invention relates to an apparatus and a method for evaluating the coverage of a test. More particularly, the present invention relates to an apparatus and a method for evaluating the coverage of a software test operating on a server when a client performs the test.
In web application development, similar to other software development, it is important to check the coverage of software testing to determine whether a sufficient test has been performed. Checking of the testing coverage is performed at the time of execution of the test and upon exit of the test.
There are various techniques and tools to check software test coverage in a white box test. However, in practice, the techniques and tools to check the coverage in the white box test are typically only applied to tests at a component level.
Further, to insure coverage of input parameters in a black box test, there are techniques that can be used such as an equivalence partitioning method, a boundary value analysis, an all-pair method, and the like. There are various tools that assist in using these techniques.
Japanese Patent Publication No. 2009-288831 describes a verification data creation technique. Initially, a value is defined for each factor type. From a plurality of test items constituted by a combination of a predetermined number of factors, a minimum value and a maximum value of a sum of values of all factors constituting the test items are calculated. A plurality of zones are generated such that a range between the minimum value and the maximum value is divided into the plurality of zones. Then, test items are generated so as to cover all combinations of factor types between two factors. Subsequently, for each of the test items, a sum of values of all factors constituting the test item is calculated, and a zone which is not covered by such a sum of values is selected as an uncovered zone. Finally, a test item is generated to cover the uncovered zone.
Japanese Patent Publication No. 2000-267888 describes measuring a software test execution coverage rate. Test point list data and test log data are input, and it is determined whether a test point recorded in the test log data is registered with the test point list data. When the test point is registered, it is assumed that a test corresponding to the test point is finished, and content of the test point list data is updated. After that, by referring to the test point list data which is finally updated, the ratio of test points which have been tested with respect to target points to be tested is calculated, and a result of the calculation is output.
Japanese Patent Publication No. 2009-301129 describes a coverage measurement device that extracts an execution path of a target control program to be tested. Then, a range of input values is specified for a function in the execution path of the control program. Subsequently, a discrete area which is a space obtained by dividing the input range by a predetermined method is set. Finally, the coverage is measured based on an execution rate of a test corresponding to the discrete area.
Japanese Patent Publication No. 2007-026360 describes a technique that includes storing input data as authorized data when the results of a verification test are considered to be appropriate. Further, input data that can be a target in a computer apparatus are collected and stored. Then, the ratio of the number of authorized data to the number of input data that can be a target in the computer apparatus is calculated as a coverage ratio, and the ratio is used for determination on overs and shorts of test cases or test data.
Further, among the techniques described in the above patent publications, there are some techniques related to the retesting of software.
For example, Japanese Patent Publication No. 2007-328447 describes a software test item selecting device which selects test items to be retested from already performed test items. Initially, in order to check whether software constituted by a plurality of modules satisfies each function in a design specification, test item running history information is given to each of a plurality of first test items, which are set beforehand. The test item running history information is associated with running history information for specifying a module of the software that is executed when a given first test item is performed. Then, based on defective module information indicative of modules including a defect and modified module information indicative of a module in which a defect is modified among the modules including a defect, a third test item is obtained. The third test item is obtained by deleting, from the test item running history information, a second test item, which is associated with running history information that includes a module in which a defect is not modified. The third test item is output.
Japanese Patent Publication No. 2008-262473 describes a retesting system. Initially, a unique component identifier parameter, and a unique input-output signal identifier for input-output operation of the component or a screen identifier given as constant information per screen unit on which the component is displayed are added to a software component. Then, along with the unique component identifier parameter, operation information at the time of performing a test, as well as display information, which is a result of the test, are stored in a predetermined place in a trace storage database, for use at the time of performing a retest of modified software.